Add sdxl controlnet reference community pipeline #9893
Conversation
dimitribarbot
changed the title
dimitribarbot
changed the title
examples/community/README.md
Outdated
|  | ||
|  |
There was a problem hiding this comment.
It seems to me that the url at https://github.com/zideliu/diffusers/assets/34944964/743848da-a215-48f9-ae39-b5e2ae49fb13 cannot be found (404). The diffusers repository fork seems to have been removed by their author. Do you confirm?
Actually, I've updated the stable_diffusion_xl_reference.py file with the latest modifications from StableDiffusionXLPipeline on my side. If you agree, I propose to create a new PR:
- Upgrading the
StableDiffusionXLReferencePipelineto use the latest modifications fromStableDiffusionXLPipeline(IP adapters...), - Replacing the human example of this updated
StableDiffusionXLReferencePipelineby a non-human one and fixing the 404 url at the same time.
There was a problem hiding this comment.
Other PR is #9938.
There was a problem hiding this comment.
Maybe you could open a PR to https://hf.co/datasets/huggingface/documentation-images and host the images there?
There was a problem hiding this comment.
Yes, PR https://huggingface.co/datasets/huggingface/documentation-images/discussions/393 has been created accordingly.
examples/community/README.md
Outdated
| Output Image | ||
|
|
||
| `prompt: 1 girl` | ||
|
|
||
| `reference_attn=True, reference_adain=True, num_inference_steps=20, style_fidelity=1.0` | ||
|
|
||
|  | ||
|
|
||
| `reference_attn=False, reference_adain=True, num_inference_steps=20, style_fidelity=1.0` | ||
|
|
||
|  | ||
|
|
||
| `reference_attn=True, reference_adain=False, num_inference_steps=20, style_fidelity=1.0` | ||
|
|
||
|  |
There was a problem hiding this comment.
Can we avoid using human references in the examples and restrict ourselves to using non-human objects?
There was a problem hiding this comment.
Sure. I've pushed modifications to fix that.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
* Add reference_attn & reference_adain support for sdxl with other controlnet * Update README.md * Update README.md by replacing human example with a cat one Replace human example with a cat one * Replace default human example with a cat one * Use example images from huggingface documentation-images repository --------- Co-authored-by: Sayak Paul <[email protected]>
What does this PR do?
It provides reference controlnet support for sdxl model used along other controlnet models. It is based on Stable Diffusion ControlNet Reference and Stable Diffusion XL Reference community pipelines, and inherits the Stable Diffusion XL ControlNet Pipeline.
Fixes #4675
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.